Handling Nonpositive Curvature in a Limited Memory Steepest Descent Method

We propose a limited memory steepest descent method for solving unconstrained optimization problems. As a steepest descent method, the step computation in each iteration only requires the evaluation of a gradient of the objective function and the calculation of a scalar stepsize. When employed to solve certain convex problems, our method reduces to a variant of the limited memory steepest descent method proposed by Fletcher (Math Prog 135(1–2):413–436, 2012), which means that, when the history length parameter is set to one, it reduces to a steepest descent method inspired by that proposed by Barzilai and Borwein (IMA J Num Anal 8:141-148, 1988). However, our method is novel in that we propose new algorithmic features for cases when nonpositive curvature is encountered. That is, our method is particularly suited for solving nonconvex problems. With a nonmonotone line search, we ensure global convergence for a variant of our method. We also illustrate with numerical experiments that our approach often yields superior performance when employed to solve nonconvex problems.